School of Information Technologies Preliminary Results on Modeling Cpu Utilization of Mapreduce Programs
نویسندگان
چکیده
In this paper, we propose an approach for predicting the CPU utilization of applications when they are running on MapReduce. Our approach has two key components: a set of an application experiments running on MapReduce to profile the CPU utilization of the application on a given platform, and a regression-based model that maps the MapReduce configuration parameters (number of Mappers, number of Reducers, size of File System (HDFS) and the size of input file) into the CPU utilization for the application. This derived model can be used for predicting CPU requirements of the same application running on MapReduce on the same platform. Our approach aims to eliminate error-prone manual processes and presents a fully automated solution. Our evaluation on running three real applications (WordCount and Exim Mainlog parsing) on pseudo-distributed mode MapReduce shows that our automated model generation procedure can effectively characterise the CPU resource of these applications with average prediction error of 3.5% and 2.75%, respectively.
منابع مشابه
On Modeling CPU Utilization of MapReduce Applications
In this paper, we present an approach to predict the total CPU utilization in terms of CPU clock tick of applications when running on MapReduce framework. Our approach has two key phases: profiling and modeling. In the profiling phase, an application is run several times with different sets of MapReduce configuration parameters to profile total CPU clock tick of the application on a given platf...
متن کاملSchool of Information Technologies Preliminary Results on Using Matching Algorithms in Map-reduce Applications
In this paper, we study CPU utilization time patterns of several Map-Reduce applications. After extracting running patterns of several applications, the patterns with their statistical information are saved in a reference database to be later used to tweak system parameters to efficiently execute unknown applications in future. To achieve this goal, CPU utilization patterns of new applications ...
متن کاملThesis Report: Resource Utilization Provisioning in MapReduce
In this thesis report, we have a survey on state-of-the-art methods for modelling resource utilization of MapReduce applications regard to its configuration parameters. After implementation of one of the algorithms in literature, we tried to find that if CPU usage modelling of a MapReduce application can be used to predict CPU usage of another MapReduce application.
متن کاملAutomatic Tuning of MapReduce Jobs using Uncertain Pattern Matching Analysis
In this paper, we study CPU utilization time patterns of several MapReduce applications. After extracting running patterns of several applications, the patterns along with their statistical information are saved in a reference database to be later used to tweak system parameters to efficiently execute future unknown applications. To achieve this goal, CPU utilization patterns of new application...
متن کاملA Study on Using Uncertain Time Series Matching Algorithms in Map-Reduce Applications
In this paper, we study CPU utilization time patterns of several MapReduce applications. After extracting running patterns of several applications, the patterns along with their statistical information are saved in a reference database to be later used to tweak system parameters to efficiently execute future unknown applications. To achieve this goal, CPU utilization patterns of new application...
متن کامل